Move C functions to pybind with GIL release #6204
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Resolves #6199
Releasing the GIL with pybind
This seem to work fine, mostly due to this being a relatively simple function, and the data inside it is guarded by the job_node mutex.
Regarding confirmed_running:
We determined that the way the refresh function was refactored, it was reading/checking for the same file over-and-over again needlessly. This way we limit the checking from when the state is
JOB_QUEUE_RUNNING
and only until the existence of the file is proven.Re-add submit+kill functions that was reverted earlier
A month ago we reverted some functions that was ported from cwrap to pybind, where the experience was a slowdown/lockdown of the whole application.
Using gil_scoped_release we can re-add these functions again, with the same premises as other code in this PR.
Pre review checklist
Ground Rules),
and changes to existing code have good test coverage.
Pre merge checklist